CEQE to SQET: A study of contextualized embeddings for query expansion

In this work, we study recent advances in context-sensitive language models for the task of query expansion. We behavior existing and new approaches lexical word-based expansion both unsupervised supervised contexts. For models, Contextualized Embeddings Query Expansion (CEQE) model. introduce a model, Supervised with Transformers (SQET) that performs as classification leverages context pseudo-relevant results. these tasks ad-hoc document passage retrieval. conduct experiments combining probabilistic retrieval well neural ranking models. evaluate effectiveness on three standard TREC collections: Robust, Complex Answer Retrieval, Deep Learning. analyze results extrinsic effectiveness, intrinsic ability to rank terms, perform qualitative analysis differences between methods. find out CEQE statically significantly outperforms static embeddings across all datasets Recall@1000. Moreover, embedding-based methods multiple collections (by up 18% Robust 31% Learning average precision) also improves over proven pseudo-relevance feedback (PRF) SQET by 6% P@20 term evaluation is approximately effective performance. Models incorporating CEQE-based score achieves gains 5% 2% AP state-of-the-art transformer-based re-ranking Birch.